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ABSTRAL  f 


We  have  N stages  to  sequentially  construct  I suc- 
cessful components.  At  each  stage,  we  allocate  a certain 
amount  of  money  for  the  construction  of  a component.  If  y 
is  the  amount  allocated,  then  the  component  constructed  will 
be  a success  with  probability  P(y)  , where  P is  a con- 
tinuous nondecreasing  function  satisfying  P (0 ) ■ 0 . After 
each  component  is  constructed,  we  are  informed  as  to  whether 
or  not  it  is  successful.  If,  at  the  end  of  the  N stages, 
we  are  i components  short,  then  a final  i T.ulty  cost  C(i) 
is  incurred.  The  problem  is  to,  at  each  stage,  determine 
how  much  money  to  allocate  so  as  to  minimize  the  total 
expected  cost  (construction  cost  plus  penalty  cost)  incurred. 
This  model  is  similar  to  that  presented  in  [1]  except  for  the 
cost  function. 

In  Section  2,  we  obtain  the  structure  of  the  optimal 
policy.  In  particular,  if  C(i+1)  - C(i)  C(i+2)  - C(i+1)  , 

and  if  y^d)  denotes  the  optimal  value  to  allocate  when  i 
components  are  needed  with  n stages  remaining,  then  y^d) 
is  nondecreasing  in  i and  nonincreasing  in  n . 

In  Section  3,  we  consider  the  special  case  of 

A 

C(i)  = iA  . If  we  now  write  y (i)  to  indicate  the  depen- 

n 

A 

dence  on  A , we  show  that  y (i)  is  nondecreasing  in  A 

n 

for  fixed  n and  i . We  also  find  that  the  necessary  and 
sufficient  condition  under  which  it  is  never  optimal  to 


construct  components  (buy  them  instead  at  a fixed  price  of 


A per  component)  is  given  by  A inf  y/P(y)  . We  then 

y 

show  that  whenever  i > n and  A > Inf  y/P(y)  , then  y (i) 

— n 

y 

la  just  the  largest  value  of  y that  minimizes 
y + (1  - P(y))A  . Finally,  we  investigate  special  forms  for 
P(y)  , namely,  P(y)  “ min(y,l)  for  y <_  1 , and 
P(y)  - 1 - e"y  . 
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OPTIMAL  SYSTEM  ALLOCATIONS  WITH  PENALTY  COSTS 


by 

C.  Deroan,  G.  J.  Lieberman  and  S.  M.  Ross 
1.  Introduction 

We  have  N stages  to  sequentially  construct  I successful 
components.  At  each  stage  we  allocate  a certain  amount  of  money  for 
the  construction  of  a component.  If  y is  the  amount  allocated  then 
the  component  constricted  will  be  a success  with  probability  P(y), 
where  P is  a continuous  nondecreasing  function  satisfying  P(0)  - 0. 
After  each  component  is  constructed  we  are  informed  as  to  whether  or 
not  it  is  successful.  If,  at  the  end  of  the  N stages,  we  are  i 
components  short  then  a final  penalty  cost  C(i)  is  incurred.  The 
problem  is  to,  at  each  stage,  determine  how  much  money  to  allocate  so 
as  to  minimize  the  total  expected  cost  (construction  cost  plus  penalty 
cost)  incurred.  This  model  is  similar  to  that  presented  in  [1]  except 
for  the  cost  function. 

In  Section  2,  we  obtain  the  structure  of  the  optimal  policy. 

In  particular  if  C(i+1)  - C(i)  < C(i+2)  - C(i+1),  and  if  y (i) 
denotes  the  optimal  value  to  allocate  when  i components  are  needed 
with  n stages  remaining,  then  y (i)  is  nondecreasing  in  i and 
nonincreasing  in  n. 
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In  Section  3,  we  consider  the  special  case  of  C(i'  LA. 

A 

If  we  now  write  yn(i)  to  indicate  the  dependence  on  A,  we  show  tr.nt 

A 

i)  is  nondecreasing  in  A for  fixed  n and  i.  We  al  o fin:  vr 

the  necessary  and  sufficient  condition  under  which  it  is  never  optimal 
to  construct  components  (buy  them  instead  at  a fixed  price  of  A per 
component)  is  given  by 


A < inf  y 7P(y)  . 

y >0 

We  then  show  that  whenever  i > n anu  A > inf  y/P(y  , then  y i' 

y n 

is  just  the  Largest  value  of  y that  minimizes  y + (1  - F(y  A. 
Finally,  we  investigate  special  forms  for  Ply',  namely,  Pty  min(y,l) 
for  y <_  1,  and  P(y)  - 1 - e y. 


Structure  of  the  Optimal  Policy 

If  we  let  ^n(i)  denote  the  minima.'  expected  additional  costs 
incurred  if  there  are  n stages  to  go  and  we  still  need  an  aliilior.a. 
i successful  components,  then  Vn(i ' satisfies  the  optimality  eq.ations 

VQ(i)  - C(i)  , i ■ 0 

V (i)  = min  {y+(l-p(y))v  .ii)+P(y)V  . Ci-D  « . 
n C(i)>y>0  n_1  n_1 

i ■ 0,  n > 1 

V°>  - 


0. 


If  we  let  ynU)  denote  the  largest  value  of  y that  minimizes  the 
above  then  the  policy  that  allocates  yn(l)  when  i components  are 
needed  with  n stages  remaining  is  an  optimal  policy. 

Lemma  1:  If  y(c)  represents  the  largest  value  of  y that  minimizes 

y + (1  - P(y))c,  then  y(c)  is  a nondecreasing  function  of  c. 

Proof:  Let  y^  = y(c^)  and  1ft  c0  - c^  + e,  t > 0.  Now 

yL  + (1  - P(y1))c1  < y + (1  - Ply))^, 
or 

y1  + (1  - P(y1))c2  - (l  - P(yL))€  < y + (1  - P(y))c2  - (l  - P(y))< 

or 

y1  + (1  - P(y1))c?  < y + (1  - Piy))c?  ■*  e(F(y)  - Fty^^) ) 

< y + (1  - Ply)  Ic-,  whenever  y < y 

implying  that  y{c2)  > y±.  | | 

We  shall  assume  throughout  that  C(i)  is  a nondecreasing 
function  of  i,  with  C(0)  - 0. 

Lemma  2:  V (i)  is  a nonincreasing  function  of  n and  a nondecreasing 

function  of  i. 

Proof ; Follows  immediately  from  the  definition  of  V (i)  and  the 
raonotonici ty  of  C(i).  || 
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Lemma  j:  If 


C(  1+1)  - C(i)  ‘_C(i+.0  - Clin),  i 0,  I,  ... 

then  the  following  inequalities  are  satisfied: 

Vl(1*U  - W1'  - V,;  ' *•  n i° 

W1!  • vn(l)  i'O11  ■ 

vli'-U  - V (1)  < v (i+?>  - v (in),  i,  n > o . 

n n — n n - 

Proof:  The  proof  is  by  induction  on  k s n+i.  As  the  inequalities 

A.  , B,  , C,  are  true  when  k - 0 assume  they  are  true  whenever 
i,n'  i,  n i,  n 

n+i  < k.  Now  suppose  n+i  - k.  We  first  show  that  A is  trie. 

l , n 

As  A,  is  clearly  valid  when  i - 0,  suppose  that  i > 0.  Now 
i,n 

for  some  y 

V A,(i)  « y + P(y)  V (i-1)  + (I  - P(y) ' V (i) 
n+1  n n 

and  thus 

Vn+l(i)  ‘ Vn(i1  " ^ + p(yHVn(i-0  - V^O]  . 

A]  so, 

vn+i(i+1)  ~y + P(y]  vn(i)  + (1  ■ ptf})  Vi+1) 

Vn+1(i+l)  - Vn(i+1)  < y + P(y)[Vn(i)  - Vn(i+1) ] . 

1* 


A.  : 
i,n 


’i,n- 


0,n‘ 


and  thus 


Thus  A,  will  foiiow  if  we  can  show  that 
i , n 

V ( i ) - V ( 1+1 ) < V (i-1)  - Vn(l)  . 
n n — n n 


However  this  is  just  the  inequality  C.  . which  is  true  when 

i+n  = k by  the  Induction  hypothesis. 

To  prove  B.  we  first  note  that  for  some  y 

i,n 


Vn+2(i)  = y + P(y}  vn+l(i-1)  + (1  ' P(y})  Vl(l) 


and  thus 

Va(1>  ’ = * * P(y)ivntl(i-ii  - vntl(i)j  . 


Also, 

Vn+l(i)  - y + Vn(i_1)  + (1  ’ P(y)}  Vn(i) 

implying  that 


Vl(i)  - Vn(i)  + p(>HVn(i-l)  - Vn(i)  ] . 


Therefore  B.  will  follow  if  we  could  prove  that 
i,n 


V1-11  - 


vn(M 


Vi*1' 


1)  - 


V ^,(1) 

n+1 


However  "he  above  is  just  the  inequality  ^ which  is  thus 
true  by  the  induction  hypothesis. 
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To  prove  C we  first  note  that  B.  , . states  that 

l,n  i+l,n-l 


2V  (i+1)  < V .(1+1)  + V^i+1) 
n — n-l  n+l 


and  thus  C.  will  follow  if  we  can  show  that 
i,n 


Vn-1(i+l)  + Vn+1(i+D  < Vn(i)  + Vn(i+2).  (1) 


Now  for  some  y 


Vn(i+2)  ‘ Vl(i+1)  *■  y + (1  ' p(y))[vn_1^i+?)  - Vn-l(i+1)] 


Furthermore, 


Vn+1(i+l)  - Vn(i)  < y + (1  - P(y))[Vn(i+l)  - Vn(i) ] 


Thus  (l)  (and  C,  ) will  follow  if  we  can  show  that 
i»n 


v (i+i)  - v (i)  < v . (i+2)  - v . (m) 

n n n-l  n-l' 


(2) 


Now,  from  A,  . it  follows  that 
i,n-l 


Vin>  -V1’  svn-i(ln)  - vn-i(1> 


and  thus  it  suffices  to  show  that 


However  the  above  Is  just  the  Inequality  C.  . which  la  true  by 

i f n*  i 

the  Induction  hypothesis.  1 1 
Theorem  1:  If 

C(l+1)  - C(l)  < C( 1+2 ) - C(i+1) 

then 

yR(l)  Is  nor.decreasl  ng  In  1 and  nonincreasing  In  n. 
Proof:  As 

Vn(i)  = mlnfy  + P(y)  V ^(1-1)  + (1  - P(y) ) V^l)) 

y 

= min(y  + (l  - P(y)  )(Vn-1(l)  - V^l-l)])  + V^d-1) 


the  result  follows  from  Lemma  1 and  the  Inequalities  C,  and  A. 

1, n 1, n 

of  Lemma  5.  1 1 


Remarks : 

(1)  The  results  go  through  in  an  Identical  manner  when  one  is  only 

allowed  to  allocate  amounts  of  money  within  a given  class  (2 

as  long  as  Inf  (y  + (l  - P(y))c)  Is  attained  for  all 

y£<7 

c > 0. 


(ii)  The  structural  result  presented  In  Theorem  1 can  be  used  to 
reduce  the  computations  needed  for  obtaining  the  optima!  p.»| 

For  instance  we  can  write 


V(i)r  min  f y + P(y )V  ( i-1)  + (1 -l’(y) )V  , ( 1) 

n / , , . / . , , n- 1 n-1 

Vn-Vi]  -y  - V l_1) 


y a 


(ill)  When  F(y)  is  concave  it  follow,  that 


y + P(y)  vn_1u-D  + (i  - P(y')vn_1(i) 

is  convex  and  thus  its  negative  is  unimodal.  This  fact  can 
be  used  to  reduce  computations  in  the  obvious  manner. 


3 . The  Special  Case  C t i ) - LA 

An  important  special  case  under  which  Theorem  1 is  valid  is 
when  C(i)  - LA.  Such  a penalty  cost  function  would  be  vali  ! if  at 
the  end  of  the  problem  we  were  forced  to  buy,  at  a price  A per 
component,  additional  components  to  make  up  for  those  we  are  short. 

It  seems  reasonable  that,  as  a function  of  A,  the  optimal 

amounts  to  allocate  should  be  nondecreasing.  That  is,  writing 

A A 

Vn(i),  yn ( i ) to  indicate  the  dependence  on  A,  then  it  seems 
intuitive  that  y^'i)  should  be  nondecreasing  in  A.  We  now  prove 


Theorem  2:  A*'  Is  nondecreaaing  in  A for  fixed  ji,  i. 


Proof:  We  will  prove  the  theorem  by  first  proving  that 


All  - / (i)  is  nonincreasing  in  A, 

n n-l 


Ai)  - Ai  -1)  is  nondecreasing  in  A. 


As  both  of  the  above  statements  are  easily  verified  when  n = 1, 
»js 'me  they  are  true  when  n = k Now 


v£+1(i)  = min  (y  + P(y)(v£(i-1)  - v£(i)]}  + vj(i) 


Implying,  by  the  induction  hypothesis,  that 


V^i(i)  - v£(i)  it  nonincreasing  in  A. 


Hence  it  remains  to  show  that  v£+^(i)  " v£+^(i-l)  is  nondecreasing 
in  A.  Now, 

v£+1(U  = min(y  + (l  - P(y))[vJ(i)  - vj(i-l)])  + vj(i-l) 
y 


and  thus,  by  the  induction  hypothesis,  it  follows  that 


<n(D  - <d-i)  is  nondecreasing  in  A , 


which  implies  from  (U)  that 


vi(1>  - - ^n*1’  - V£<1-1>  + <(1-1>  - v£+i<1-1) 


9 


1.  nondecreaaing  In  A.  Thua  the  atatementa  (3)  are  proven  by 
induction.  The  theorem  now  follows  from  Lemma  1 and  the  second  in- 
equality In  (3)  aince 

/(!).. in  (,  * II  - P(y))l»*1.1(ll  11 

n y >0 

Another  question  of  interest  Is  when  it  is  optimal  to  always 
invest  0.  That  la,  under  what  conditions  on  P(y)  is  it  optimal 
never  to  attempt  to  construct  components  yourself  but  rather  to  buy 
them  at  a fixes  price  of  A per  component?  If  we  let  Un(i)  denote 
the  return  from  the  policy  of  always  investing  0 then  U^i)  will 
satisfy  the  optimality  equation  if 

u (i)  < y ♦ P(y>  t ,(1-D  * <1  - P(y>>  Vi(1)  for  1,11  y ' 0 

n — 

which,  since  Un(i)  - iA  is  equivalent  to 

LA  < y + p(y)  ( i-l)A  (l-  P(y))lA  for  ail  * 0 

or,  equivalently,  if 

, „ y for  all  y > 0 

k-n7) 

or 

a - r>o  ■ 
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Since,  in  finite  stage  dynamic  programming  problems,  a policy  is 
optimal  if  and  only  if  its  return  ‘xtisfies  the  optimality  equation 
it  follows  that  (5)  is  the  necessary  and  sufficient  condition  for  it 
to  be  optimal  to  always  invest  0. 

When  1 > n and  A > inf  y/P(y)  then 

y 

y ( i ) " largest  value  of  y that  minimizes  y + (1  - P(y))A. 

The  above  follows  by  noting  that  when  i 2 n the  problem  1b  equivalent 
to  n separate  problems  each  of  which  is  such  that  if  you  invest  y 
and  are  successful  then  your  cost  is  y and  if  you  are  unsuccessful 
then  your  cost  is  y + A. 

Remarks: 

(a)  In  the  special  case 

P(y ) = min(y, 1)  , y < 1 

it  is  easy  to  verify  that 

0 If  A < 1 

1 if  A > 1 

(b)  In  the  special  case 

P(y)  - 1 - e'y 

11 
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It  can  be  verified  that  the  V (1)  and  y (i)  can  be 

n n 

recursively  computed  according  to 

Vn(1)  " 1 + yn(l)  + Vn-l(i-1)'  1 > 1.  n > i 

where 

yn(i)  - logiv^d)  - vn_1(i-i)],  1 > 1,  n > i 

using  the  boundary  condition 


V (0)  » 0 . 

n 
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